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style. However, the application of this technique is difficult because it involves non-tnvial 
human input. This paper presents a novel framework for performing assume-guarantee 
reasoning in an incremental and fully automated fashion. To check a component against a 
property, our approach generates assumptions that the environment needs to satisfy for the 
property to hold. These assumptions are then discharged on the rest of the system. 
Assumptions are computed by a learning algorithm. They are initially approximate, but 
become gradually more precise by means of counterexamples obtained by model ch ecking 
the component and its environment, alternately. This iterative process may at any stage 
conclude that the property is either true or false in the system. We have implemented our 
approach in the LTSA tool and applied it to the analysis of a NASA system. 
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Abstract. Compositional verification is a promising approach to addressing the state explo- 
sion problem associated with model checking. One compositional technique advocates prov- 
ing properties of a system by checking properties of its components m an assume-guarantee 
style However, the application of thisjechnique is difficult because!! mvolvesnon-mvial hu- 
man input. This paper presents a novel framework for performing assume-guarantee reason- 
ing in an incremental and fully automated fashion. To check a component against a property, 
our approach generates assumptions that the environment needs to satisfy for the property to 
hold These assumptions are then discharged on the rest of the system. Assumptions are com- 
puted by a learning algorithm. They are initially approximate, but become gradually more 
precise by means of counterexamples obtained by model checking the component and ifr en- 
vironment, alternately. This iterative process may at any stage conclude that t he property is 
either true or false in the system. We have implemented our approach m the LTSA tool and 
applied it to the analysis of a NASA system. 


1 Introduction 

Our work is motivated by an ongoing project at NASA Ames Research Center on the application of 
model checking to the verification of autonomous software. Autonomous software involves co 
P t_ 8 Manors to, reacting to external stimuli w.thom hunxm miervemton. Extensive 
verification is a pre-requisite for the deployment of missions that mvolve autonomy 

Given some formal description of a system and of a required property, model checking auto 
maticallv determines whether the property ts satisfied by the system If the property 

it returns a counterexample, i.e., an execution of the system that ^eedl to store 

limitation of the approach, referred to as the “state-explosion problem 8], is that it needs to 
the explored system states in memory, which is impossible for most realistic systems. 

Compositional verification presents a promising way of addressing state **^*'*Z 

ca[esTP‘divideTmdTftrtquer”~approach where properties of the system-are deeomposed-mto prop- 

erties of its components, so that if each component satisfies its respective property, then so does the 
am therefore model ehecked separately. I. is often the case, how.veh 
that components only satisfy properties in specific contexts (also called environments). This 
yen rise to the assume-guarantee style of reasoning [18, 21]. 


* This author is grateful for the support received from RIACS to undertake this research while participating 
in the Summer Student Research Program at the NASA Ames Research Center. 
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Assume-guarantee reasoning first checks whether a component M guarantees a property P , 
when it is part of a system that satisfies an assumption A. Intuitively, A characterizes all contexts 
in which the component is expected to operate correctly. To complete the proof, it must also be 
shown that the remaining components in the system, i.e., M’s environment, satisfy A. This style 
of reasoning is typically performed in an interactive fashion. Developers first check a component 
under no assu mp tions about the environment. If a counterexample is returned that is unrealistic 
for the system under analysis, they make several attempts at manually defining an assumption that 
is strong enough to eliminate false violations, but that also reflects appropriately the remaining 
system. 

This paper presents a novel framework for performing assume-guarantee reasoning in an incre- 
mental and fully automatic fashion. Our approach iterates a process based on gradually learning 
assumptions. The learning process is based on queries to component M, and on counterexamples 
obtained by model checking M and its environment, alternately. Each iteration may conclude that 
the required property is satisfied or violated in the system analyzed. This process is guaranteed to 
termin ate; in f act, it conver ges to an as sumpti cm th^ is negessary and sufficient for the property, to 
hold in the specific system. 

Our approach has been implemented in the Labeled Transition Systems Analyzer (LTS A) [20], 
and applied to the analysis of the Executive module of an experimental Mars Rover (K9) devel- 
oped at NASA Ames. We are currently in the process of also implementing it in Java Pathfinder 
(JPF) [23]. In fact, as our approach relies on standard features of model checkers, it is fairly 
straightforward to add in any such tool. 

The remainder of the paper is organized as follows. We first provide some background in 
Section 2, followed by a high level description of the framework that we propose in Section 3. The 
algorithms that implement this framework are presented in Section 4. We discuss the applicability 
of our approach in practice and extensions that we are considering in Section 5. Section 6 describes 
our experience with applying our approach to the Executive of the K9 Rover. Finally, Section 7 
presents related work, and Section 8 concludes the paper. 


2 Background 

The presentation of our approach is based on techniques for modeling and checking concurrent 
programs implemented in the LTS A tool [20]. The LTS A supports Compositional Reachability 
Analysis (CRA) of a software system based on its architecture, which, in general, has a hierarchical 
structure. CRA incrementally computes and abstracts the behavior of composite components based 
on the behavior of their immediate children in the hierarchy [13]. The flexibility that the LTS A 
provides in selecting any component in the hierarchy for analysis or abstraction makes it ideal for 
■experimenting" with out approach: 


Labeled Transition Systems. The LTSA uses Labeled Transition Systems to model the behavior 
of communicating components in a concurrent system. Let Act be the universal set of observable 
actions and let r denote a local action unobservable to a component’s environment. We use i r 
to denote a special error state , which models the fact that a safety violation has occurred in the 
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Fig. 1. Example LTSs 
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associated system. We require that the error state has no outgoing transitions. Formally, a Labeled 
Transition System (LTS ) Misrs four tuple {Q, aM, 5, go) where: 


- Q is a set of states 

- aM C Act is a set of observable actions called the alphabet of M 
_ S CQx {aM U {r}} x Q is a transition relation 

- qo € Q is the initial state 


We use n to denote the LTS ({tt}, Act, 0, ir) . An LTS M = (Q, aM 5, go) is non-determimstic 
if it contains r-trans.tions or if 3 (g, a, q'), (g, a, g") £ 5 such that q' * g". Otherwise, M is deter- 

Consider a simple communication channel that consists of two components whose LTSs are 
shown in Fig. 1. Note that the initial state of all LTSs in this paper is state 0. The Input LTS 
receives an input when the action input occurs, and then sends it to the Output LTS with ac- 
tion send. After some data is sent to it, Output produces output using the action output an 
acknowledges that it has finished, by using the action ack. At this point, both LTSs return to their 
initial states so the process can be repeated. 


Traces. A trace a of an LTS M is a sequence of observable actions that M can perform starting 
at its initial state. For example, (input) and (input, send) are both traces of the Input LTS 
in Fig 1 For E C Act, we use a f E to denote the trace obtained by removing from a all 
occurrences of actions a g 27- The set of all traces of M is called the language of M, denoted 
C(M). 


Parallel Composition— We-provide transitionaLsemantics for parallel composition-in a tv-pic 

process algebra style, although our aim here is not to define an algebra. Let M - (Q, a T, , qo) 
and M’ = l O' aM' ,5' qk). We say that M transits into M' with action a, denoted M — * M , 
if and only if (g 0l a, go) € <5 and either aM = aM' and 5 = 5' for q' 0 # n, or, in the special case 

where q[\ — tt , M l = U. .. 

The parallel composition operator || is a commutative and associative operator that : combines 

the behavior of two components by synchronizing the actions common to their alphabets an 
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interleaving the remaining actions. For example, in the parallel composition of the Input and 
Output components from Fig. 1, actions send and ack will each be synchronized. 

Formally, let M, = (Q 1 ,aM 1> S 1 , q%) and M 2 = {Q 2 , aM 2 , S 2 , q$) be two LTSs. If M : = U 
or M 2 = 17, then M x || M 2 = II. Otherwise, Mi || M 2 is an LTS M = {Q,aM,S,q 0 ), where 
Q = Q 1 x Q 2 , q 0 = (?o,?o)> = a ^ 1 U a ^ 2, 311(1 5 1S defined “ follows (note that the 

symmetric rules are implied by the fact that the operator is commutative): 

Mi M[, a ^ a M 2 M\ — — > M[, M 2 — » a ^ r 

Mi || M 2 M[ || M 2 Mi || M 2 M[ || M’ 2 

Properties. We call a detenninistic LTS that contains no tt states a safety LTS. A safety property 
is specified as a safety LTS P, whose language C (P) defines the set of acceptable behaviors over 
aP. An LTS M satisfies P, denoted as M j= P, if and only if V<r £ C (M) : (a f aP) € C (P). 

When checking a property P, an error LTS denoted P err is created, which traps possible 
violations. with the_7r state....Formally, the. error LTS of a property P = {Q,_aP, <5, go) is P err = 
{<3 U { 7r } j ocP err , S', qo }, where aP err = aP and 

S' = 6 U {(<?, a, tt) I a € aP and Jg' € 5 : (q, a , g') 6 <5} 

Note that the error LTS is complete , meaning each state other than the error state has outgoing 
transitions for every action in the alphabet. 

For example, the Order property shown in Fig. 2 captures a desired behavior of the commu- 
nication channel shown in Fig. 1. The property comprises states 0, 1 and the transitions denoted 
by solid arrows. It expresses the fact that inputs and outputs come in matched pairs, with the 
input always preceding the output. The dashed arrows illustrate the transitions to the error 
state that are added to the property to obtain its error LTS. 

To detect violations of property P by component M, the parallel composition M || P err is 
computed. It has been proved that M violates P if and only if the 7 r state is reachable in M || 
Perr [5]. For the example system, state 7 r is not reachable in Input [| Output || Order err , so 
Input || Output |= Order. 


Assume-Guarantee Reasoning. In the assume -guarantee paradigm a formula is a triple {A} M (P), 
where M is a component, P is a property and A is an assumption about M’s environment [21]. 
The formula is true if whenever M is part of a system satisfying A, then the system must also 
guarantee P. 

The LTSA is particularly flexible in performing assume-guarantee reasoning. Both assump- 
tions and properties are defined as safety LTSs 3 . In fact, a safety LTS A can be used as an assump- 
tion or as a property. To be used as an assumption for module M, A itself is composed with M, 
thus playing the role of an abstraction ofATs environment. To“be used as a properfy"to"becheCked 
on M , A is turned into A err and then composed with M. 

To check an assume-guarantee formula (A) M (P), where both A and P are safety LTSs, the 
LTSA computes A || M || P err and checks if state tt is reachable in the composition. If it is, then 
(A) M (P) is violated by component M, otherwise it is satisfied. 


3 Any LTS without tt states can be transformed into a safety LTS by determinization. 
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Deterministic Automata and Safety LTSs. One of the components of our framework is a learn- 
ing algorithm that produces Deterministic Finite-State Automata, which our framework then uses 
as safety LTSs. A Deterministic Finite-State Automaton (DFA) M is a five tuple { Q , aM, o,q 0 , ) 

where: 

- Q is a finite set of states 

- aM C Act is a set of observable actions that make up the alphabet of M 

__ S : Q x aM — ► Q is a transition function 

- q 0 £ Q is the initial state 

- F C Q is the set of accepting states 

For a DFA M and a string a, we use 5(q, a) to denote the state that M will be in after reading a 
starting at state g. A string <r is said to be accepted by a DFA M = (Q, aM, 5, go F) if %o, a) G 
F The language accepted by M, designated £ (M) is the set {a | S(qo,a) G F). 

The DFAs returned by the learning algorithm in our context are complete , minimal, and prefix- 
" ctoTdfa^automaton M fs prefiiTdosed if £TM)Ts^refix-closed, i.e.. for every rr € £ (Mp every 
prefix of a is also in £ (M)). These DFAs therefore contain a single non-accepting state. They can 
easily be transformed into safety LTSs by removing the non-accepting state, which corresponds to 
state 7 T of an error LTS, and all transitions that lead into it. 


3 Framework for Incremental Compositional Verification 

For simplicity, let us consider the case where a system is made up of two components, Mi and M 2 . 
As mentioned in the previous section, a formula (A) M (P) is true if, whenever M is part o a 
system satisfying A, then the system must also guarantee property P The simplest composition 
proof rule shows that if {A} Mi ( P ) and (true) M 2 {A) hold, then (true) M, || M 2 (P) is true. 
This proof strategy can also be expressed as an inference mle as follows: 

(Step 1) (A) Mi (P) 

(Step 2) {true) M 2 (A) 

(true) Mi || M 2 (P) 

Note that this rule is not symmetric in its use of the two components, and does not support cir- 
cularity. Despite its simplicity, our experience with applying compositional verification to sever 
applications has shown it to be the most useful mle in the context of safety property checking. 

For the use of the compositional mle in proving (true) Mj || M 2 (P) to be justified, the 
assumption must be more abstract than M 2 . An appropriate assumption for the mle needs to be 
strong enough for M ; to satisfy P in Step 1. Moreover, the restrictions it plac es on M 2 should 
reflect M 2 ’s behavior. Coming up directly with an appropriate assumption for the applicationm 
the compositional mle is a non-trivial process. So in practice, the mle is typically applied in an 
iterative fashion as illustrated in Fig. 4. At each iteration t, an assumption - , is provi e ase on 
some knowledge about the system and on the results of the previous iteration. A model checker 
can then be used to automatically apply the two steps of the compositional mle. 

Step 1 is applied first, to check whether Mi guarantees P in environments that satisfy A { . It the 
result is false, it means that this assumption is too weak, i.e.. A, does not restrict the environment 
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Fig. 4. Incremental compositional verification during iteration i 


enough for P to be satisfied. The assumption therefore needs to be strengthened (which corre- 
sponds to removing behaviors from it) with the help of the counterexample produced by Step 1. In 
the context o f the next assump tion A j+i , component M\ should n ot e xhibit the violating behavior 
reflected by this counterexample, at least. 

If Step 1 returns true, it means that is strong enough for the property to be satisfied. To 
complete the proof, Step 2 must be applied to discharge on M2. If Step 2 returns true, then 
the compositional rule guarantees that P holds in M\ || M2. If it returns false, further analysis is 
required to identify whether P is indeed violated in M\ || M2 or whether A{ was stronger than 
necessary. Such analysis is usually based on the counterexample returned by Step 2. 

If assumption A t is too strong it must be weakened (i.e., behaviors must be added) in iteration 
i 4. 1. The result of such weakening will be that at least the behavior that the counterexample 
represents will be allowed by assumption A i+ v The new assumption may of course be too weak, 
and therefore the entire process must be repeated. 

The bottleneck in the application of the above process lies in the fact that coming up with 
and refining assumptions manually tends to be a mental challenge. To address this issue, we have 
developed a framework that implements this iterative, incremental process in a fully automated 
way. A learning algorithm, described in detail in the following section, is used for assumption 
generation. 

4 Algorithms 
4.1 The L* Algorithm 

The learning algorithm used by our approach was developed by Angluin [3] and was later im- 
proved by Rivest and Schapire [22]. In this paper, we will refer to the improved version by the 
name of the original algorithm, L*. L* learns an unknown regular language and produces a DFA 
That accepts ifTCeTC^be an unknown regulaflanguage over some“alphabet P .“In ord SFtcrl e arrrf/ , 
L* needs to interact with a Minimally Adequate Teacher , from now on called Teacher. A Teacher 
needs to be able to answer correctly two types of questions from the algorithm. The first type is a 
membership query , consisting of a string a €P*\ the answer is true if a e U, and false otherwise. 
The second type of question is a conjecture , that is, a candidate DFA C whose language C(C) the 
algorithm believes to be identical to U. The answer is true if C (C) = U . Otherwise the Teacher 
returns a counterexample, which is a string o in the symmetric difference of C (C) and V . 
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(1) let S = E = {A} 
loop { 

(2) Update T using queries 
while (S, E , T) is not closed { 

(3) Add sa to 5 to make S closed where 5 E 5 and a 6 £ 

(4) Update T using queries 

(5) Construct candidate DFA C from (£, E, T ) 

(6) Conjecture C is correct 
if C is correct 

(7) return C 
else 

(8) Add e € E* that witnesses the counterexample to E 

} 


Fig. 5. The L* Algorithm 


At a higher level, L* creates a table where it incrementally records whether finite strings in 
E* belong to U . It performs this by making membership queries to the Teacher. At various stages 
during this process, L* decides that it is ready to make a guess. It constructs a candidate automaton 
C based on the information contained in the table, and asks the Teacher whether the conjecture is 
correct. If it is, the algorithm terminates. Otherwise, L* uses the counterexample returned by the 
Teacher to extend the table with strings that witness differences between L (C) and U. 

In the following more detailed presentation of the algorithm, line numbers refer to L* s illus- 
tration in Fig. 5. L* builds an observation table (S, E, T) where 5 and E are a set of prefixes 
and suffixes, respectively, both over 27*. In addition, T is a function mapping (5 U S ■ 27) • E to 
{true, false}, where the operator is defined as follows. Given two sets of event sequences P 
and Q, P ■ Q = {pq | p € P and q € Q}, where pq represents the concatenation of the events se- 
quences p and q. Initially, L* sets 5 and E to be {A} (fine 1), where A represents the empty string. 
Subsequently, it updates the function T by making membership queries so that it has a mapping 
for every string in (5 U 5 • 27) ■ E (fine 2). It then checks whether the observation table is closed, 
i.e., whether 

Vs € 5,Va € 27,3s' € 5,Ve G E : T(sae) = T(s'e) 

If (S, E, T) is not closed, then sa is added to S where s e S and a € E are the elements for 
which there is no s' e S (line 3). Once this has been added to S, T needs to be updated (line 4). 
Lines 3 and 4 are repeated until (5, E, T ) is closed. . 

Once the table is closed, a candidate DFA C = ( Q , otC , 6, qo,F) is constructed (line ), wi 
states (5 = 5, initial state <j 0 = A, and alphabet aC = E (27 is the alpha bet of the unknown 
lancnmg p U) The set of final states F are the states s e 5 such that T(s) = true. TheTransition 
relation 5 is defined as 6(s,a) = s' where Ve e E : T(sae) = T(s'e). Such an s' is guaranteed 
to exist when (S,E,T) is closed. The candidate C is presented as a conjecture to the Teacher 
(line 6). If the conjecture is correct, i.e., if C. (C) = U, the L* Algorithm returns C as correct 
(line 7) otherwise it receives a counterexample c € 27* from the Teacher. 

The counterexample c is analyzed by L* to find a suffix e of c that witnesses a difference 
between £ (C) and U ; e must be such that adding it to E will cause the candidate to reflect this 
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difference 4 (line 8). Once e has been added to E, the L* Algorithm iterates the entire process by 
looping around to line 2. 


Characteristics of L*. Let M be the minimal automaton such that C (M) = U . The L* algonthm 
is guaranteed to terminate with M as its last conjecture. The characteristic of L* that makes it 
particularly attractive for our work is that at any stage, the automata that it generates are minimal. 
In other words, if L* makes a conjecture C based on observation table (5, E, T), then any other 
DFA C that is consistent with ( S,E y T ) but not equivalent to C contains more states than C. In 
addition, the conjectures made by L* strictly increase in size; each conjecture is smaller than the 
next one, and all incorrect conjectures are smaller than M. If M has n states, L* makes at most 
n- 1 incorrect conjectures. The number of membership queries made by L* is O ( kn 2 + n log m) , 
where k is the size of the alphabet of the language U , n is the number of states in the minimal DFA 
for U y and m is the length of the longest counterexample returned when a conjecture is made. 


4.2 Learning for Assume-Guarantee Reasoning 

Assume a system Mi || M 2 , and a property P that needs to be satisfied in the system. In the 
context of the compositional rule presented in Section 3, the learning algorithm is called to guess 
an assumption that can be used in the rule to prove or disprove P . An assumption with which the 
rule is guaranteed to return conclusive results is the weakest assumption A w , which restricts the 
environment of M\ no more and no less than is necessary for P to be satisfied. Assumption A w 
describes exactly those traces over E = (aMiUaP)OaM 2 which, when simulated on M\ || P err 
cannot lead to state 7 r. The language C (A w ) of the assumption contains at least all traces of M 2 
abstracted to E that prevent Mi from violating P. 

Formally, A w is such that, for any environment component Me, {true) M 1 || Me (P) if and 
only if (true) M e (A w ) [14]. In our framework, L* learns the traces of A w through the itera- 
tive process described in Section 3. The process terminates as soon as compositional verification 
returns conclusive results, which is often before the weakest assumption A w is computed by L*. 

For L* to learn A w , we need to provide a Teacher that is able to answer the two different kinds 
of questions that L* asks. Our approach uses model checking to implement such a Teacher. 


Membership Queries. To answer a membership query for o = (a i, 02 , ■ • * > a n) in E* the 
Teacher simulates the query on Mi || P. For clarity of presentation we will reduce such simu- 
lations to model checking, although we have implemented them more efficiently, directly as simu- 
lations. So for string cr, the Teacher first builds A a = (Q, aA a) 5 y q 0 ) where Q = {g 0 ,*? 1 , • • • > 
ctA a = E y S = | 0 < i < n}, and q 0 = q°. The Teacher then model checks 

(Ao) Mi (P). If true is returned, it means that cr 6 C (A w ), because Mi does not violate P in the 

contextT)f _ ir7so _ 'the _ TeacherretnmstraerOtherwiserthe"answerto-the“membership-query is false. 


Conjectures. Due to the fact that in our case the language C (A w ) that is being learned is prefix- 
closed, all conjectures returned by L* are also prefix-closed. Our framework transforms these 
conjectures into safety LTSs (see Section 2), which constitute the intermediate assumptions A*. 


4 The procedure for finding e is beyond the scope of this paper, but is described in [22]. 
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In our framework, the first priority is to guide L* towards a conjecture that is strong enough 
to make-Step 1 of the compositional rule return true. Once this_is accom£hshed,_t e res g 
conjecture may be too strong, in which case our framework guides L* towards a conjecture tha 
is weak enough to make Step 2 return conclusive results about whether the system satisfies \ 
The way the Teacher that we have implemented reflects this approach is by using two Oracles and 
Counterexample Analysis to answer conjectures as follows. 

- Oracle 1 performs Step 1 in Fig. 4, i.e„ it checks (A,) Mi (P). If this does not hold the 
model checker returns a counterexample c. The Teacher informs L* 'that its conjecture A, is 
not correct and provides c \ E to witness this fact. If, instead, (A,) Mj (P) holds, the Teacher 

forwards A % to Oracle 2. t . , . , , 

- Oracle 2 performs Step 2 in Fig. 4 by checking (true) M 2 (A,). If the result of model chec - 
ing is true, the teacher returns true. Our framework then terminates the compositional veri- 
fication because P has been proved on Mi || M 2 (according to the compositional rale). If 
model checking returns a counterexample, the Teacher performs some analysis to distinguish 

the underlying reason (see Section 3 and Fig. 4). . , , ~ OT , 

- Counterexample Analysis is performed by the Teacher in a way similar to that used for an- 
swering membership queries. Let c be the counterexample returned by Oracle 2. The Teacher 
computes A c ,s and checks (A ctr ) Mi (P). If true, it means that Mi does not violate P 
in the context of c, so c \ L is returned by the Teacher as a counterexample for conjecture 
A . If the model checker returns false with some counterexample c , it means that P is vio- 
lated in Mi || Ms, so there is no need for a more precise assumption to be generated Our 
framework then appropriately combines c with c' in order to generate a counterexample for 

(true) M\ || M 2 {P}* 


4t 3— Example 

Given components Input and Output as shown in Fig. 1 and the property Order shown in Fig. 2, 
we will check (true) Input || Output (Order) by using our approach. The alphabet of the as- 
sumptions that will be used in the compositional rale is E = {(alnputUaOrder) naOu pu ) 

^ S S As^ described, attach iteration L* updates its observation table and produces a candidate as- 
sumption whenever the table becomes closed. The first closed table obtained is shown m Table 1 
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Table 1. Mapping T\ 



Ei 


T\ 

A 

Si 

A 

true 


output 

false 


ack 

true 


output 

false 

Si • E 

send 

true 


output, ack 

false 


output, output 

false 


output, send 

false 


Table 2. Mapping T 2 



Ei 


t 2 

A ack 


A 

true true 

s 2 

output 

false false 


send 

true false 


ack 

true true 


output 

false false 


send 

true false 


output, ack 

false false 

S 2 * E 

output, output 

false false 


output, send 

false false 


send, ack 

false false 


send, output 

true true 


send, send 

true true 


and its associated assumption, Ai, is shown in Fig. 6. The Teacher answers conjecture A\ by first 
invoking Oracle 1, which checks (Ai) Input (P). Oracle 1 returns false, with counterexample 
a = (input, send, ack, input), which describes a trace in A\ || Input || P err that leads to 
state 7r. 

The Teacher therefore returns counterexample a \ E = (send, ack) to L*, which uses 
queries to update its observation table until it is closed. From this table, shown in Table 2, the 
assumption A 2 . shown in Fig. 7, is constructed and then conjectured to the Teacher. This time, 
Oracle 1 reports that (A 2 ) Input (P) is true, meaning the assumption is not too weak. The Teacher 
calls Oracle 2 to determine if (true) Output (A 2 ). This is also true, so our algorithm reports that 
(true) Input || Output (P) holds. 

This example did not involve weakening of the assumptions produced by L*, since the assump- 
tion A 2 was sufficient for the compositional proof. This will not always be the case. For example, 
let us substitute Output by Output ' illustrated in Fig. 3, which allows multiple send actions to 
occur before producing output. The process will be identical to the previous case, until Ora- 
cle 2 is invoked by the Teacher for conjecture A 2 - Oracle 2 returns that (true) Output (A 2 ) is 
false, with counterexample (send, send, output). The Teacher analyzes this counterexample 
and determines that in the context of this trace, Input does not violate P. The trace is returned to 
the L* Algorithm, which will weaken the conjectured assumption. The process involves two more 
iterations, dunng which assumptions A3 (Fig. 8) and A4 (Fig. 9), are produced. Using assump- 
tion A4, which is the weakest assumption A w , both Oracles report true, so our assume-guarantee 
framework reports that (true) Input || Output 1 (P) holds. 


5 Discussion 

5.1 Correctness 

Theorem 1. Given components Mi and M 2 , and property P, the algorithm implemented by our 
framework terminates and it returns true if P holds on Mi || M 2 and false otherwise. 
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Proof. To prove the theorem we will first argue correctness of our approach, and then the fact that 
it terminates. 

- Correctness. The Teacher in our framework uses the two steps of the compositional mle to 
answer conjectures. It only returns true when both steps return hue, and therefore correctness 
is guaranteed by the compositional rule. Our framework reports an error when it detects a trace 
<j of M 2 which, when simulated on Mi, violates the property, which implies that Mi || M 2 
violates P. 

- Termination. At any iteration, our algorithm returns true or false and terminates, or continues 
by providing a counterexample to L*. By correctness of the L* Algorithm we are guaranteed 
that if L* keeps receiving counterexamples, it will eventually, at some iteration t, produce A w . 
During this iteration, Step 1 will return true by definition of A w . The Teacher will therefore 
apply Step 2, which will return either true and terminate, or a counterexample. This counterex- 
ample represents a trace of M 2 that is not contained in L(A W ). Since, as discussed m Section 
4, A„ is both necessa ry and sufficient, analysis of the counterexample will return false, and 
the algorithm will terminate. 


5.2 Practical Considerations 

In our context, the languages queried by the L* Algorithm are prefix-closed. This is because 
our technique applies to purely safety properties; any finite prefix of a trace that satisfies such 
a property must also satisfy the property. Therefore, when for some string a a membership query 
L\\ Mi (P) returns false, we know that for any extension of a the answer will also be false. We 
can thus improve the efficiency of the algorithm by reducing the cost of some of the membership 
queries that are answered by the Teacher. For example, in the observation table shown m Table , 
the entry for (output) is false. The Teacher can return false for the quenes (output, ack), 
(output, send), and (output, output) without invoking the model checker. 

In the framework presented, membership queries, conjectures and counterexample analysis all 
involve model checking, which is performed on-the-fly. The assumptions that are used m these 
steps are increasing in size, and grow no larger than the size of A w . In our experience, for well- 
designed systems, the interfaces between components are small. Therefore, assumptions are ex- 
pected to be significandy smaller than the environment that they represent in the composition^ 
rules. Although the L* algorithm needs to maintain an observation table, this table does not need 

to be kept in memory while the model checking is performed. 

Note that our framework provides an “any time” [11] approach to compositional verification. 
If memory is not sufficient to reach termination, intermediate assumptions are generated, which 
may be useful in approximating the requirements that a component places on its environment in 
order to satisfy certain properties. 


5.3 Extensions 

Generalization. Our approach has been presented in the context of two components. Assume 
now that a system consists of n components M l || - • • || M n . The simplest way to generalize 
our approach is to group these components into two higher level components, and apply the com- 
positional rules as already discussed. Another possibility is to handle the general case without 
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computing the composition of any components directly. Our algorithm provides a way of check- 
ing (true) Mi || M 2 ( P ) in a compositional way. If M 2 consists of more than one component, 
our algorithm could be applied recursively for Step 2. This is an interesting future direction, in 
particular since the membership queries concentrate on a single component at a time. However, 
we need to further investigate how meaningful such an approach would be in practice. 

Computing the Weakest Assumption. The L* Algorithm can also be used to learn the weak- 
est possible assumption A w that will prevent a component Mi from violating a property P. This 
assumption will be generated without knowing M 2 , the component Mi interacts with. The only 
place in our assume-guarantee framework where M 2 is used is in Oracle 2, when the Teacher tries 
to determine if the Assumption generated is too strong. Oracle 2 can be replaced by a confor- 
mance checker, for example the W-Method [6], which is designed to expose a difference between 
a specification and an implementation. This will produce a set of sequences that are guaranteed to 
expose an error in the conjectured assumption if one exists. This approach to learning the weakest 

assumptionJs. an anytime algorithim[ll]._The sequence_ofintermediate_assumptions.conjectured 

by the teacher are approximate and become more refined the longer the L* Algorithm runs. As 
discussed previously, an approximate assumption can still be useful. 


6 Experience 

We implemented the assume-guarantee framework described above in the LTSA tool, and exper- 
imented with our approach in the analysis of the executive subsystem for the K9 Mars Rover, 
developed at NASA Ames. The executive receives flexible plans from a Planner, which it executes 
according to the plan language semantics. A plan is a hierarchical structure of actions that the 
Rover must perform. For increased autonomy, each action is associated with a number of state or 
temporal pre-, maintenance, and post-conditions, which must hold before, during, and on comple- 
tion of the action execution, respectively. 

The executive has been implemented as a multi-threaded system, where synchronization be- 
tween threads is performed through mutexes and condition variables. The developer provided 
design documents for several versions of the system. These documents described the synchro- 
nization between components in an ad-hoc flowchart-style language. They looked very much like 
LTSs, which allowed us to translate them in a straightforward and systematic way into the input 
language of the LTSA. 

One of the properties described by the developer refers to a subsystem of the executive consist- 
ing of two components: -the main coordinating component named “Executive”, and a component 
responsible for monitoring state conditions named “ExecCondChecker” The property' places the 
following requirement on this subsystem, irrespective of the behavior of the subsystem’s environ- 
ment: for a specific variable of the ExecCondChecker shared with the Executive, if the Executive 
reads the value of the variable, then the ExecCdhdCH^ker shduld not readTEs value befdrelhe 
Executive clears it first. 

We used our compositional verification framework to check this property on one of the newer 
versions of the model. We set Mi = ExecCondChecker and M 2 = Executive. The expenment 
was conducted on a Pentium III 500 MHz with 1 Gb of memory running RedHat Linux 7.2 using 
Sun’s Java SDK version 1.4.0-01. To check the property directly by composing the two modules 
with the property required searching 3,630 states and 34,653 transitions and took 0.535 seconds. 
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Table 3. Results of the Rover Example 


Iteration 

EH 

States 

Transitions 

Result 

1 - Oracle 1 

1 

5 

24 

Too weak 

2 - Oracle 1 

2 

268 

1,408 

Too weak 

3 - Oracle 1 

3 

235 

1, 209 . 

Too weak 

4 - Oracle 1 

5 

464 

2, 500 

Not too weak 

4 - Oracle 2 

5 

32 

197 

Incompatible 


Table 3 shows the results of using our assume-guarantee framework on this example, which 
took 8.639 seconds. The |Ai| column gives the number of states of the assumptions generated. 
The States and Transitions columns give the number of states and transitions explored during the 
analysis of the assumption and the Result column gives the result of the analysis. When using 
Oracle 1 to determine if the Assumption was too weak, iterations 1-3 all determined that the 
assumption was-too weak. -In iteration 4, Oracle 1- determined that the learned assumption. was.noL 
too weak. The assumption was then given to Oracle 2 that returned a counterexample which, when 
simulated on the ExecCondChecker, led to an error state. Thus, the assume-guarantee analysis 
concluded that the property does not hold. The largest analysis occurred when using Oracle 1 
during iteration 4, and this required exploring fewer states than checking the property directly. 

We also used the L* Algorithm to generate the weakest assumption that the ExecCondChecker 
make s on the Executive for the property described to be satisfied in this subsystem. The method 
described in [14] needed 24.623 seconds to generate the 6 state weakest assumption. The L* Al- 
gorithm, using the W-Method for Oracle 2, took 9.530 seconds to generate the 6 state assumption, 
although conformance testing kept running after that since it could not yet conclude if this was the 
weakest possible assumption. 

Assumptions for Java Programs. We are currently working on an implementation of our ap- 
proach in Java PathFinder (JPF) [23], a model checker for Java programs developed at NASA 
Ames. We have already developed a prototype with the assumption generation algorithm based on 
the W-Method, and have experimented with it on several Java programs. To generate assumptions 
for Java programs, no changes needed to be made to JPF, but we needed to add a method call at 
program points where actions of interest occurred. The Teacher calls JPF to answer both quenes 
and conjectures. 

7 Related Work 

One way of addressing both the design and verification of large systems is to use their natural 
decomposition into compone nts. Formal techniq u es for support of component-based de _ si gnare_ 
gaining prominence, see for example [9, 10]. In order to reason formally about components m 
isolation, some form of assumption (either implicit or explicit) about the interaction with, or inter- 
ference from, the environment has to be made. Even though we have sound and complete reasoning 
systems for assume-guarantee reasoning, see for example [7, 16, 18, 21] and more recently [24], it 
is always a mental challenge to obtain the most appropriate assumption [17]. 

It is even more of a challenge to find automated techniques to support this style of reasoning. 
The thread modular reasoning underlying the Calvin tool [12] is one start in this direction. In the 
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framework of temporal logic, the work on Alternating-time Temporal Logic ATL (and transition 
systems) [1] was proposed for the specification and verification of open systems together with au- 
tomated support via symbolic model checking procedures. The Mocha toolkit [2] provides support 
for modular verification of components with requirement specifications based on the ATL. 

In previous work [14], we presented an algorithm for automatically generating the weakest 
possible assumption for a component to specify a required property. Although the motivation of 
that work is different, the ability to generate the weakest assumption can also be used to auto- 
mate assume -guarantee reasoning. A disadvantage of that algorithm is that it does not compute 
partial results, meaning no assumption is obtained if the computation runs out of memory. This 
may happen if the state-space of the component is too large. Our approach generates assumptions 
incrementally and may terminate before A w is computed. Moreover, even if it runs out of memory 
before reaching conclusive results, intermediate assumptions may be used to give some indication 
to the developer of the requirements that the component analyzed places on its environment. 

The problem of generating an assumption for a component is similar to the problem of gener- 
ating component. interfaces, to deal with intermediate, state explosion imCRA^Several. approaches 
have been defined for automatically abstracting a component’s environment to obtain interfaces [4, 
19]. These approaches do not address the issue of incrementally refining interfaces, as needed for 
carrying out an assume-guarantee proof. 

Groce et al have used the L* Algorithm for Adaptive Model Checking [15]. In this work, 
the goal is to learn a model of a software system which can then be given to a model checker. 
This approach is similar to our approach for learning assumptions using a conformance checker 
and the L* Algorithm. To determine if the model accurately describes the system, a conformance 
checker is used, which is expensive in terms of time. Until an accurate model is obtained, this 
process cannot be used to show that a property is satisfied and can only be used to help the analyst 
discover a counterexample. In our approach, an inaccurate assumption can still be used to complete 
an assume-guarantee proof. 


8 Conclusions 

Although theoretical frameworks for sound and complete assume-guarantee reasoning have ex- 
isted for decades, their practical impact has been limited because they involve non-trivial human 
interaction. In this paper, we presented a novel approach to automating such reasoning in an incre- 
mental fashion. Our approach uses a learning algorithm to generate and refine assumptions based 
on queries and counterexamples, in an iterative process. The process is guaranteed to terminate, 
and return true if a property holds in a system, and a counterexample otherwise. If memory is 
not sufficient to reach termination, intermediate assumptions are generated, which may be useful 
in approximating the requirements that a component places on its environment in order to satisfy 
certain properties. 

One advantage of our approach is its generality. It relies on standard features of model check- 
ers, and could therefore easily be introduced in any such tool. Moreover, the architecture of our 
framework is modular, so its components can easily be substituted by more efficient ones. It re- 
mains to be shown, of course, how useful our approach will prove in practice. However, our early 
experiments with real case studies provide strong evidence in favor of this line of research. 

In the future we plan to investigate a number of topics including whether the learning algo- 
rithm can be made more efficient in our context; whether different algorithms would be more 
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appropriate for generating the assumptions; whether we could benefit by querying a component 
and its environment at the same time, or by implementing more powerful compositional rules. An 
interesting challenge will also be to extend the types of properties that our framework can handle 
to include liveness, fairness, and timed properties. 
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